
s .° c 'f 


Advances in Engineering Education \QQ 


The Skyline TEAMS Model: A Longitudinal Look at the 
Impacts of K-12 Engineering on Perception, Preparation 
and Persistence 

MALINDA S. ZARSKE 
University of Colorado Boulder 

JANET L. YOWELL 
University of Colorado Boulder 

HEIDI L. RINGER 
Skyline High School 
Longmont, CO 

JACQUELYN F. SULLIVAN 
University of Colorado Boulder 

and 

PATRICIA A. QUINONES 
Skyline High School 
Longmont, CO 


ABSTRACT 


This paper describes the longitudinal impacts of a partnership between the University of Colorado 
Boulder’s K-12 Engineering Education initiative and the St. Vrain Valley School District. Together, 
university and high school educators created a replicable pre-college engineering model in a nine- 
school feeder system, which serves many Colorado students who are traditionally underrepresented 
in the engineering profession, and culminates with a high school STEM (science, technology, engineer¬ 
ing, and mathematics) Academy whose graduates are motivated to thrive in engineering colleges. 
However, the following question, “Is this an effective model for increasing student STEM persistence 
and performance?” remains a driver for our investments as we refine the K-12 engineering program 
based on partner school feedback and quantitative and qualitative assessment results. 

Data show that our K-12 engineering program has positively impacted student perception, pre¬ 
paredness, and persistence in engineering based on regular, pre-college hands-on engineering de¬ 
sign experiences. A description of the multi-year K-12 engineering model and longitudinal analysis 
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of students’ concurrent math course choices is presented. Also, subsequent student persistence in 
and from the STEM Academy is addressed. 


TRENDS IN K-12 ENGINEERING 


According to the U.S. Census Bureau, one half of the U.S. population by 2050 will be nonwhite [1], 
This means that engineering solutions will have to be acceptable to an increasingly diverse popu¬ 
lation, and the engineering profession will have to draw much more heavily on underrepresented 
groups for the country to maintain, let alone increase, its technological capability [2], 

While the overall number of engineering degrees granted in the U.S. has gone up in recent years, 
although not to its historic high of almost 80,000 in 1985 [2], just 4.5% of our nation’s BS gradu¬ 
ates earn engineering degrees, masking the reality that only 1.6% of all bachelor level degrees are 
awarded to women in engineering [3, 4], As women make up an increasingly higher majority of 
all college graduates, the system resistence to expanding women’s participation in engineering is 
alarming; begging the question of who will engineer our processes and products of the future? By 
comparison, throughout Asia, 21% of university graduates are engineers, while in Europe, 12.5% of 
university graduates earn engineering degrees [2], 

The persistently low student achievement among high poverty students is not an acceptable 
baseline condition for the engineering profession—and is a barrier to broadening participation. To 
help close achievement gaps, engineering offers a real-world application of the fundamental sci¬ 
ence and math principles that students explore early in their education. Schoolroom experiences 
in an engineering context provide a gateway to creative, real-life problem solving, and exploration 
in all areas of science and mathematics. By experiencing the engineering design process first hand, 
students begin to see how engineering ingenuity and inventions touch and shape their everyday 
lives, and they begin to develop a self-identity that possibly encompasses a science, technology, 
engineering, and mathematics (STEM) future. 

An engineering education creates access to the American dream and should provide people 
from all backgrounds, regardless of circumstance, the opportunity to share in that dream. However, 
African Americans and Hispanics collectively accounted for only 11% of BS engineering graduates 
in the U.S. in 2009 [5], And, despite strong K-12 preparation, the participation of women in the en¬ 
gineering and technology pipeline has declined since 1995, now accounting for 18.1% of engineering 
bachelor’s degrees awarded in 2010, up slightly from 17.8% in 2009 [5], And, sadly, the technology 
gender gap is wide: while girls comprised 49% of high school students taking the AP calculus exams 
in 2011, only 21% of teens who took AP computer science exams were female [6], Clearly, as society 
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becomes more technology-driven, the representation of women interested in contributing to the 
technological revolution is shrinking. 

Together, our education system and contemporary society poorly arm youth with the tools and 
interest to succeed and compete in STEM fields, discouraging them from joining the ranks of tomor¬ 
row’s scientists and engineers. And, while today’s students show decreased inclination to choose 
science, engineering and technology futures, our nation’s need for product and process innovations— 
the work of engineers—has never been more compelling. Further, research reveals that K-12 students 
are not considering math and science as professionally relevant—tragically, closing off future career 
pathways as early as third grade [7], Continuing on this path will greatly diminish our state and na¬ 
tion’s capacity to perform, boding poorly for our nation’s productivity and future. Only with bold, 
replicable strategies to augment the preparation of high-potential, disadvantaged K-16 students can 
engineering colleges more broadly serve the profession and our nation’s students. 


REVIEW OF LITERATURE ON HANDS-ON ENGINEERING IN K-12 


Engineering design at the pre-college level has exploded through the introduction of nationwide 
competitions and partnerships with local engineering colleges. In 2006, the National Academy of 
Engineering (NAE) and the National Research Council responded to the increase in K-12 engineer¬ 
ing initiatives by forming the Committee on K-12 Engineering Education to investigate the learning 
of engineering in K-12 settings and what, if any, best practices exist for K-12 engineering instruction 
and learning [8], The resulting 2009 publication, Engineering in K-12 Education: Understanding the 
Status and Improving the Prospects, analyzed myriad engineering curricula, the science of learning 
engineering in the K-12 setting, and evidence supporting the effectiveness of engineering in the K-12 
arena. The investigation articulated a need for more research around potential impact on students 
from practicing engineering at the K-12 level; however, they did find evidence of improved learning 
and achievement in science and mathematics, as well as an increased awareness of engineering as 
a career [8] from early experiences. 

K-12 engineering work is often grounded in the existing research on inquiry- and project-based 
learning, whose popularity reflects research advances on cognitive development in the fields of 
neuroscience and psychology. The research findings propose that learning is partly a social and 
cultural activity, and learners access and use their prior knowledge to explore, construct, and create 
new knowledge [9]. Students engaged in project-based instruction in the classroom demonstrate 
somewhat increased content knowledge, with even deeper conceptual understanding and ability 
to extend the skills to other situations [10], Evidence also shows that a project-based instructional 
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method provides a motivating environment for the teaching of basic skills, increased student cog¬ 
nition with more complex problems, as well as student exhibition of higher professional skills and 
creativity, than students who are taught traditionally [9, 10, 11], As a result, the K-12 engineering 
community has tailored project-based engineering design experiences for K-12 audiences [11], 

Research at CU-Boulder focusing on the retention of engineering students who have exposure 
to a hands-on, project-based design course in their first year of undergraduate study shows an 
overall 64% retention rate into the seventh semester (compared to a 54% retention rate of students 
not enrolled in the course), with an even higher retention of women and minority students [12,13], 
Similarly, analysis of hands-on engineering design activities in the K-12 setting has demonstrated 
an increase in students’ STEM content knowledge and interest in engineering [14], Although only 
a small number of K-12 students go on to pursue engineering careers, the broad exposure to engi¬ 
neering at the K-12 level can lead to more technologically-literate citizens and hopefully increased 
numbers and diversity of engineers [14,15], 

Throughout twelve years of executing and refining a K-12 engineering education program within three 
school districts, CU-Boulder’s K-12 engineering education work has employed accepted longitudinal 
tracking methods to determine benefits to participants, long-term project outcomes, and impact on 
the partnering institutions [16,17]. We have found that involvement in the K-12 engineering education 
partnerships supports students’ enrollment in more rigorous courses of study in high school, and—in 
many cases—affected their college decisions (i.e., whether or not to pursue a collegiate engineering 
education) and, ultimately, their career path [14]. Furthermore, engagement of high school students 
in a multi-year, university-sponsored program can also inspire allegiance to that university [14], 


TOMORROW’S ENGINEERS...CREATE. IMAGINE. SUCCEED. PROGRAM 


In 2004, the Integrated Teaching and Learn¬ 
ing Program in the University of Colorado Boul¬ 
der’s College of Engineering and Applied Sci¬ 
ence identified the student populations of the 
Lafayette, Colorado-area schools and the Denver 
School of Science and Technology as the focus 
of its GK-12 engineering initiative. Their goal, to 
improve science and math literacy and increase 
the number of high school students prepared 
to choose an engineering and/or technology 



Fifth grade TEAMS graduate engineering fellow 
teaches elementary students how engineering 
can help lessen the impacts of natural disasters. 
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college path, led to the TEAMS Program and partnership—Tomorrow’s Engineers... creAte. iMagine. 
Succeed. Their approach in the TEAMS initiative was to partner with high-need schools to make 
engineering exploration part of every child’s educational experience in grades 3-12, through weekly, 
in-class, hands-on, inquiry-based engineering instruction, augmented with summer programming 
C http://itll.colorado.edu/index.php/k-12 enqineering/tomorrows engineers ). 

After success in Lafayette, in fall 2009 they implemented the TEAMS program in the St. Vrain 
School District, in Longmont, CO—partnering with six elementary, two middle and one high school 
in a district that has a diverse student population and a strong desire to increase the number of 
students pursuing a STEM future. For two years prior, the TEAMS Program collaborated with district 
and school officials to set the stage for this opportunity, as they restructured to become STEM focus 
schools in response to the planned opening of a new high school, a few miles away, in a high-income 
community. To help reduce the high number of students, particularly white students, who would 
leave the school (as predicted by district officials) and open enroll in the new high school, or drop 
out of high school completely, the Skyline High School STEM Academy program was created to 
prepare students for career opportunities of the future as well as develop students’ 21 st century skills. 
Further, thinking was that a “white flight” demarcation in demographics would deny all students 
the rich, diverse cultural experience and education that Skyline High offered its students. Today, the 
work to prevent “white flight” appears to have worked: the high school population is comprised of 
-45% free/reduced lunch and -52% minority students—mirroring the demographic makeup of the 
surrounding community. And, CU’s TEAMS Program support of Skyline High School and its eight 
feeder schools aligns directly with the high school’s mission for increasing the number of under¬ 
represented students who enter the STEM higher education pipeline. 

The TEAMS model digs deep into the district’s feeder system, engaging over 1,900 students an¬ 
nually in all nine schools in the Skyline High feeder system, priming students from 5 th through 8 th 
grade to take full advantage of the high school STEM Academy. This systems approach is intended 
to significantly increase the number of graduates who enter engineering college and other STEM 
programs—and is designed to be replicated in other high schools across the nation. 


THE SKYLINE HIGH SCHOOL STEM ACADEMY 


Skyline High School appears at first glance to be a below average American high school. Located 
in Longmont, Colorado, and one of the St. Vrain Valley School District’s ten high school options, it is 
a diverse school with 1,230 students across grades 9-12. Recent composite ACT test scores, required 
by state mandate to be taken by all 11 th grade students, for the Skyline students are 17.7 (2009), 18.9 
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(2010), and 17.5 (2011), compared to the 2011 district wide and statewide averages of 20.4 and 19.9. 
Likewise, 45% of Skyline’s students are from economically disadvantaged families (free and reduced 
lunch eligible), compared to a high school average in the district of 20% and state of 40%. 

However, a closer look shows that Skyline High School is not average at all. In 2009, Skyline began 
an innovative STEM Academy ( http://shs.stvrain.k12.co.us/stem.htm ) to help curtail the exodus of 
academically-oriented students and increase student accomplishment and graduation of students 
at risk. Believing that the “E” in STEM should not be a silent vowel, the Skyline leadership team 
partnered with the long-established TEAMS program at the University of Colorado Boulder. One of 
the goals of the STEM Academy is to mirror the demographics of the school population, ensuring 
that the program is serving their entire population—including minority youth and girls, both under¬ 
represented in STEM-related fields, and low-income youth from all backgrounds. The Skyline STEM 
Academy opened in fall 2009 with 80 freshman and 12 sophomores, and grew to 249 students in 
fall 2011, comprised of 35% female, 35% minority, and 23% F/R lunch students across grades 9-12. 

The first four-year STEM Academy cohort to graduate will do so in 2013. Evidence that the STEM 
Academy is moving Skyline students beyond average level of academic preparation are the 2011 
school wide CSAP results (Colorado’s “No Child Left Behind” assessment test), which reveal that 
while the majority of Skyline’s 9 th and 10 th grade students scored only partially proficient in math, 
writing and science, the average STEM Academy student scored proficient in every one of these 
content areas—success by any measure, and this achieved so early in the program’s tenure. 



Figure 7. The St. VrainVaiiey School District’s Skyline Feeder System. 
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Skyline’s feeder elementary and middle schools mirror the demographics of Skyline High School 
and are a key component in the recruitment and retention of underrepresented populations to the 
Academy (see Figure 1). The TEAMS program was introduced into all eight feeder school’s cur¬ 
riculum, inspiring Skyline’s leadership to develop shared learning goals between the nine schools— 
including the high school. To achieve this goal and ensure that all schools remain invested, regular 
Feeder Principal Meetings have been conducted for the past three years, and outreach to all feeder 
schools by Skyline’s leadership team has been a priority to create a sense that all students in the 
feeder system will enroll at Skyline High School. (This has not been the case in the past, as many 
students—perceiving that Skyline High School had a lack of rigor, too much focus on discipline in¬ 
stead of academics, and low expectations for their students —opted out of the feeder system and 
enrolled in one of the district’s alternate high schools.) 

Building Momentum 

To increase community awareness of the Skyline STEM Academy, the Skyline leadership team 
(and a host of enthusiastic students) frequently conducts tours of the newly renovated school for 
Parent/Teacher Organization groups, participates in community events at the feeder schools, and 
collaborates with the feeder schools on their grant proposals. Plans also include hosting elementary 
and middle school workshops in the summer to increase the commitment and expectation that 
students enroll at Skyline High School and, hopefully, become part of the STEM Academy. The high 
school also hosts a STEM Lecture Series throughout the academic year featuring guest speakers 
prominent in local STEM industries; several elementary and middle school classes regularly attend 
the presentations. Also, Skyline invites all feeder schools to bring students to their engineering de¬ 
sign expo, held at the end of each semester to showcase the projects that STEM Academy students 
create during their engineering design classes. 

Implementation of the Skyline STEM Academy 

Students are given the opportunity to apply to the high school STEM Academy during their 8 th 
grade year. Middle school visits (even to those schools outside the feeder system) are conducted in 
December, and an Open House, hosted at Skyline High School, allows students and parents inter¬ 
ested in the STEM Academy to see first-hand the new facility and meet with teachers and students 
about the program. Students apply in December, and include in their application recommendations 
from teachers who can speak to their STEM abilities and academic motivation. 

Through several iterations, and building off of the successful TEAMS Program teach-the-teacher 
model, the combined Skyline and TEAMS leadership team developed a comprehensive education 
plan (summarized in Table 1) for the high school Academy, with the overarching goal of students 
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Subject 

Description 

Years 

Required 


Core Requirements 


Math 

Must successfully complete a minimum of Pre-Calculus/ 

Trigonometry by graduation. 

4 

Science 

At least one full credit must be earned each year; required: at least one year of 
biology, chemistry, and physics. Must complete at least one full year of any AP 
science course. 

4 

Single Foreign 
Language 

Three years of one world language or two years each of two separate world 
languages. 

3 


STEM Academy Requirements 


STEM Elective- 
WIRED 

Required course for all STEM Academy 9th graders; can also be taken 
the summer prior to 9th grade. This course provides students with a basic 
introduction to Internet and technology resources such as Google Docs® and 
Power Point®. 

0.5 

STEM Elective- 
Computer 
Science 

Can be taken anytime during the 4 years: Introduction to Programming, 
Introduction to Computing, Computer Science or AP Computer Science. 

0.5 

Explorations in 
STEM 

(No Prerequisite) 

This course provides students with a foundation in the topics and skills necessary 
for success in upper-level STEM courses. Required course during freshman 
year (either semester) 

0.5 

Creative 

Engineering 

(Prerequisite: Explorations in STEM) 

Students take one full year (two classes) during their sophomore year. Course 
options are evolving, and so far include: Robotics, Assistive Technologies, 
Sustainable Design, and Structural Design. 

1 

Advanced 

Engineering 

(Prerequisite: Creative Engineering) 

Students take one full year (two classes) during their junior year. Course options 
are evolving, and so far include: Robotics, Biomedical Engineering, Engineering 
Science, and Applied Software Engineering. 

1 

Senior Capstone 
Design Course 

(Prerequisite: Advanced Engineering) 

Students take one full year during their senior year. 

1 

Other STEM 
Electives 

Any additional math, science, computer science, or STEM class beyond 
requirements. 

1 


Table 1: Graduation requirements of Skyline High School students completing the 


STEM Academy certificate program. 


pursuing post-secondary STEM degrees. In order to receive a STEM certificate upon graduation, 
students are required to take 28 total high school credits (minimum district graduation requirement is 
24.5 credits) and follow an increasingly challenging education plan during all four years in the STEM 
Academy. Students must attain grades of C or better in each of their core and STEM courses, and 
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graduate with at least a 2.5 unweighted GPA to receive the 
STEM certificate. Additionally, students must still meet all 
graduation requirements mandated by the school district. 

A key component of the STEM certification program is 
four full years of math and science for all STEM students, 
with the requirement of completing Algebra 1 by 9th 
grade. To ensure that students are prepared to succeed 
in Algebra 1, Skyline and its two feeder middle schools 
committed to the implementation of a CTAG (Closing the 
Achievement Gap) grant, provided by the Colorado De¬ 
partment of Education, for the past three years. The grant 
led to a change in Skyline’s master schedule to provide 
three tiers of intervention for those students entering 9 th 
grade at the Algebra 1 level. For example, students in Tier 
1 are prepared for Algebra 1 at 9 th grade, while students 
in Tier 2 are less prepared, and are concurrently enrolled 
in both Algebra I and an additional math lab to address 
specific holes in their math learning. Students in Tier 3 take an intensive Algebra 1 course every 
day (double the time of a regular Algebra 1). Thus, every student who enters 9 th grade deficient in 
Algebra 1 is met with a plan to boost their math achievement. Because of these math tier system 
interventions, the Algebra 1 failure rate at Skyline High School has already been reduced from 38% 
to 16%—a remarkable outcome. 

Also, on average, 25% of 9 th graders—more than those coming in at the Algebra 1 level—take 
Geometry or Algebra 2. Pushing the educational innovations lower, both feeder middle schools 
now use the Tier 2 model with their 7 th and 8 th grade classes to address holes in math preparation 
before students get to high school. 

As can be seen in Table 1, three full years of STEM engineering courses are required to earn a STEM 
Academy certificate. Each of the courses has been developed in collaboration with CU-Boulder’s 
College of Engineering and Applied Science, and provides students with fundamental engineering 
design principles and experiences, arming them with the resources for a successful first-year engi¬ 
neering program at the college level. Students begin the engineering sequence with exploring the 
engineering design process and the importance of teamwork in engineering. 

As students grow through the sequence of courses from Explorations in STEM (in grades 9-10) to 
Senior Capstone Design, they are engaged in increasingly complex hands-on design projects that 
peer into a variety of engineering disciplines. The curriculum uses a modified First-Year Engineering 



High school STEM Academy 
students work with 5 th grade client 
(a local elementary student with a 
disability) in understanding her 
needs for an improved water 
fountain device. 
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Projects course curricula, as instructed at CU-Boulder and described in “Improving Engineering 
Student Retention through Hands-On, Team Based, First-Year Design Projects,” by Knight, et al. 
[13]. Each of the engineering courses culminates in an Engineering Design Expo, where the high 
school students present their final products and display posters to an audience of engineers, school 
district officials, and K-12 students from other schools. To view Skyline STEM Academy students 
displaying their assistive technology products for a local elementary school client (see videos at: 
http://spot.colorado.edu/-schaefem/pageAEE.html ). 

Several other courses have been developed to meet the Academy goals. The 9 th grade WIRED 
course prepares students to operate in a digital academic environment and connects them to 
technology in a project-based environment. Students begin building an eFolio that will house 
their STEM work throughout their four years in the Academy, learn how to prepare a professional 
presentation—a skill that they will hone in all STEM classes—research STEM careers, and begin a 
“College in Colorado” profile. 

The Senior Capstone Design course provides students with the opportunity to research and design 
a year-long engineering project, and employ the engineering design process while working in a team. 
These projects may also be part of the Innovation Center, where students take R&D ideas from local 
businesses and develop a project with the goal of taking the idea to production or the marketplace. 

Interestingly, one of the required computer science courses, Introduction to Programming, de¬ 
veloped prior to founding of the Academy, is one of the most popular courses at the school, with 
141 students enrolled, of which 40% are Latino. 

Students in the STEM Academy also aid in the district-wide collaboration to align curriculum 
between the feeder elementary and middle schools with the high school. For example, STEM 
Academy students actively participate in the STEM Lecture Series—a community event that brings 
feeder middle and elementary students to the high school to learn alongside older students. During 
these lectures, STEM Academy students assist with the hosting of the lectures and regularly solicit 
thoughtful, relevant questions of guest speakers. 


RESEARCH QUESTIONS AND ASSESSMENT METHODS 


Is a multi-year, comprehensive STEM Academy an effective model for increasing student per¬ 
ception and persistence into STEM careers? The goal of this research is on creating a systemic 
high school STEM experience that is based on current knowledge of skills necessary for success 


10 


SUMMER 2012 






ADVANCES IN ENGINEERING EDUCATION 

The Skyline TEAMS Model: A Longitudinal Look at the Impacts of K-12 
Engineering on Perception. Preparation and Persistence 



in undergraduate STEMdisciplines. Specifically, this research paper addresses student perception, 
preparedness, and persistence in engineering for students in a public high school who have regular, 
pre-college hands-on engineering design experiences. 

Everything worth doing is worth measuring. The STEM Academy program is assessed on a con¬ 
tinuous evaluation cycle, with quantitative and qualitative methods used to assess the program’s 
success in meeting measurable objectives and outcomes. Each assessment method is selected to 
help determine if program objectives have been met. 

Quantitative methods include student attitude surveys; Academy application analyses; collec¬ 
tion of school demographic data; and student attendance and retention comparisons. Following 
the successful assessment of the TEAMS program prior to working with the Skyline STEM Academy, 
longitudinal tracking of participant course choices, grades, and transcripts are also employed. Quali¬ 
tative methods include open-ended survey questions for students; small focus groups for students 
per semester; and weekly discussions with teachers, though these are not the focus of this paper 
and will be elaborated on in future publications. 

Thus far, information has been gathered on 103 students who participated during the inaugural 
year of the STEM Academy (2009-10), and 168 students who participated during the second year 
(2010-11). This includes -350 matched pre-to-post semester attitude surveys over the two academic 
years (the number reflects 10th and 11th grade students who have taken more than one semester- 
long course), and eight small (5-6 students each) focus groups in each of the upper-level engi¬ 
neering design courses. More than three times that number of STEM Academy applications have 
been analyzed to provide information on the characteristics of students attracted to this type of 
program. Also, attendance, and retention comparisons have been made to the student population 
of the school (1,251 students in 2009-10 and 1,230 students in 2010-11). 


IMPACTS OF THE STEM ACADEMY 


Student Perception of Engineering 

The results of 103 9 th and 10 th grade students, participating in four sections of the 2009-10 Ex¬ 
plorations in STEM course and 168 9 th through 11 th grade students during the 2010-11 Explorations in 
STEM and Creative Engineering courses, provide some insight into the impact of the STEM Academy 
on student interest and awareness of engineering. After a brief introduction to engineering careers 
at the beginning of the semester, the Explorations course engages students in three or four multi¬ 
week, project-based engineering explorations guided by the engineering design process, ending 
with design prototype completion. The Creative Engineering courses engage students in a more 
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in-depth semester-long engineering design project. These classes are taught by three high school 
science teachers with prior experience and training in engineering design through CU-Boulder’s 
teacher professional development. In addition, a CU TEAMS engineering graduate student helps 
instruct the classes weekly, providing student support on their design projects. 

Students were given engineering attitude surveys about their semester-long Explorations course 
experience with choices on a five-point Likert-type scale ranging from “strongly agree” to “strongly 
disagree,” pre- and post-semester, to measure changes in student attitudes towards engineering 
as a result of exposure to engineering design. The survey instruments were modified from instru¬ 
ments already in use by CU-Boulder to evaluate their First-Year Engineering Projects course. The 
survey questions probe student attitudes and include awareness of engineering careers as well as 
interest in learning more about engineering. The surveys are paired pre- to post- for each student. 
T-tests are used to separately determine the significance of pre- to post-matched survey scores for 
awareness of engineering careers and interest in engineering. Mean scores are presented here and 
have been adjusted to a five-point Likert-type scale. 

This 2009-10 survey data contains responses to fourteen questions that queried about engineering 
careers (eight items) and interest in learning more about engineering (six items). The scores were 
summed separately from the careers questions and interest questions to get two aggregated scale 
scores for awareness and interest. For example, a higher pre-survey score for interest indicates a 
student’s greater initial interest in learning more about engineering. 

The pre- to post-mean scores of the overall 2009-10 Explorations course (n=103) demonstrate 
a significant gain from the pre-assessment (mean=4.15) to the post-assessment (mean= 4.47), p < 
0.01, for engineering awareness, indicating that the pre- to post-gains are real, and the Explorations 
course increased student awareness of engineering as a career. The mean scores of these students 
(n = 104) with respect to interest in engineering relay a gain from the pre-assessment (mean = 3.81) 
to the post-assessment (mean = 3.88); however, T-test statistical procedures indicate that this gain 
is not statistically significant (p = 0.35). 

Gender Differences 

These results, separated by gender, provide more insight into the interest and awareness gains of 
STEM Academy students, although it should be noted that sample sizes are fairly small for this type 
of analysis. With respect to gender, the mean scores of awareness and interest for both females and 
males in the inaugural cohort of the STEM Academy are shown in Table 2. Female students reported 
similar gains in engineering awareness over the semester as their male counterparts, and greater 
gains in their interest in engineering. Calculated gains are significant for engineering awareness but 
not for interest in engineering. 
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Variable 

N 

Pre Survey 
Mean (SD) 

Post Survey 
Mean (SD) 

Mean 

Difference 

Engineering Awareness 





All 

103 

4.15(0.44) 

4.47 (0.49) 

0.32* 

Female 

34 

4.06 (0.46) 

4.57 (0.32) 

0.51* 

Male 

69 

4.19(0.43) 

4.43 (0.55) 

0.24* 

Interest in Engineering 





All 

103 

3.81 (0.69) 

3.88 (0.78) 

0.07 

Female 

34 

3.77 (0.57) 

3.96 (0.63) 

0.19 

Male 

69 

3.86 (0.75) 

3.85 (0.84) 

-0.01 


** Significant at the p < 0.05 level, paired t-test 


Table 2: Preliminary results of the Skyline High School inaugural 
2009-2010 STEM Academy cohort. 

A 1-5 Likert-scale adjusted mean is given, pre then post, with standard 
deviations in parentheses. 


The preliminary results and early patterns were promising in support of hands-on high school 
design courses and indicated that more research is needed to evaluate the impact of the STEM 
Academy on student perceptions and attitudes. This led to a refinement of the surveys for the 2010- 
11 academic year in order to look more closely at student attitudes with respect to self-efficacy in 
engineering and identity with the engineering profession. 

Digging Deeper 

Initial analysis confirms that 9 th , 10 th and 11 th grade student perceptions of engineering continued 
to increase in the Skyline STEM Academy during the 2010-11 academic year (n = 168). The new survey 
questions include factors to measure how the students identify with the group of engineers (12 items) 
and self-efficacy around engineering (11 items). The latter includes questions around interest in engi¬ 
neering (I think engineering is interesting ) and confidence to succeed in engineering (/ can succeed in 
an engineering curriculum). These factors were confirmed through a Principal Components Analysis. 

Again, mean factor scores were used and correspond to a five-point Likert-type scale. For ex¬ 
ample, a higher average pre-survey score for identity indicates a student’s greater initial identity 
with their engineering peers. The surveys were then paired pre- to post- for each individual. T-tests 
were used to determine the significance of pre- to post-matched survey scores for awareness of 
engineering careers and interest in engineering separately. 
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Variable 

N 

Pre Survey 

Mean (SD) 

Post Survey 
Mean (SD) 

Mean 

Difference 

Engineering Identity 





All 

168 

4.03 (0.61) 

4.14(0.61) 

0.11* 

Female 

57 

4.04 (0.52) 

4.07 (0.53) 

0.03 

Male 

111 

4.03 (0.66) 

4.18(0.64) 

0.15* 

Engineering Self-Efficacy 





All 

168 

4.22(0.51) 

4.29 (0.53) 

0.04* 

Female 

57 

4.05 (0.54) 

4.16(0.49) 

0.11* 

Male 

111 

4.31 (0.47) 

4.35 (0.55) 

0.04* 


** Significant at the p < 0.05 level, paired t-test 

Table 3: Results of the Skyline High School fall 2010 STEM Academy cohort. 

A 1-5 Likert-scale adjusted mean is given, pre then post, with standard deviations in 


parentheses. 


The pre- to post-mean scores of STEM Academy engineering students for fall 2010 (n = 168) 
demonstrate a significant gain from the pre-assessment (mean = 4.03) to the post-assessment 
(mean = 4.14), p < 0.05, for engineering identity. And, the mean scores of these students (n = 168) 
with respect to efficacy in engineering relay a gain from the pre-assessment (mean = 4.22) to the 
post-assessment (mean = 4.29), p < 0.05 (presented in Table 3). 

These results, separated by gender, provide more insight into the evolving perceptions of the STEM 
Academy students. With respect to gender, Table 3 gives the mean scores of engineering identity 
and engineering self-efficacy for both females and males during fall 2010, for the entire STEM Acad¬ 
emy cohort. As shown, female students did not demonstrate similar gains in engineering identity 
to their male counterparts over the course of the semester. Also, female students showed greater 
overall gains in engineering self-efficacy (p < 0.05) but still did not match the self-efficacy gains of 
their male counterparts. The male students in the STEM Academy reported a higher self-efficacy 
in engineering than female students both pre- and post-semester. While, independent samples t- 
tests confirmed that the pre- and post-survey scores for engineering identity were not statistically 
different between females and males, the pre- and post-scores for engineering self-efficacy were 
indeed different (p = 0.002 and p = 0.27, respectively). 

Student perceptions of engineering, including engineering identity and engineering self-efficacy 
are changing for students in the STEM Academy. While we continue to survey students every se¬ 
mester during their tenure in the Academy, we hope to specifically answer how their perceptions 
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are evolving, both by grade and demographics. Coupled with thoughtful qualitative analysis, this 
research is poised to inform the engineering education research community on the value of early 
and frequent exposure to engineering. 

Student Preparedness for Success in Engineering Careers 

Students take the application process seriously, writing thoughtful essays and highlighting their 
academic accomplishments in middle school. And, a definite measure of success is that from 2009-11, 
Skyline saw an 11% increase in the number of students enrolling in the school from charter schools 
or middle schools outside of Skyline’s two feeder middle schools. And, an added measure of suc¬ 
cess: students are staying in the Skyline feeder system who would have otherwise gone to the IB 
program at another high school in the district. The K-16 team is encouraged that such intentional 
diversity through excellence can be a model for other school settings. 

The average GPA of STEM Academy student’s is greater than the average of the overall school, 
as shown in Table 4. As of spring 2011, the STEM weighted GPA was 3.30, while the overall school 
(grades 9-11 only for accurate comparison) had a weighted GPA of 2.65. 

Retention in the program from year-to-year is also promising. For the current 11 th graders, 77% 
(70/91) of the cohort are still in the program, as are 99% (75/76) of current 10 th graders. From the 
initial cohort of current 12 th graders, 47% (9/19) are still in the program—even though they have 
not experienced a full four years of the Academy, which is believed to be instrumental in year-to- 
year retention. 

Changing School Culture 

Although the inaugural class of STEM Academy students will not graduate for another year, the 
school has already seen an increase in the number of students participating in STEM-related activities. 
Three years ago there were no computer or engineering classes being offered at the high school, 


Grade 

STEM GPA 

School GPA 

9 

3.35 

2.57 

10 

3.27 

269 

11 

3.15 

2.70 

Overall 

3.30 

2.65 


Table 4: Spring 2011 average STEM Academy student 


CPA vs. overall school GPA. 
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and now Skyline has four teachers (2.33 FTE) 
teaching 28 sections of computer and engineer¬ 
ing courses. And, now their growing Computer 
Science and Engineering Program is a testament 
to student interest. 

Skyline STEM students also successfully part¬ 
ner with several post-secondary groups, offering 
skills to help students succeed as they pursue 
STEM undergraduate degrees (see box at right). 

Two groups of students have been working on 
out-of-class projects with professors from Uni¬ 
versities in Oregon and Arizona. Another four 
students have been hired by a local business 
partner, RidgeviewTel Wireless, employing skills they learned at Skyline through STEM Academy 
courses. 

Student Persistence: Attendance and Retention of STEM Students 

Over the last few years, the attendance at Skyline High School has steadily improved. In 2008- 
09, a 90% attendance rate, coupled with an astonishing 142 suspensions, was the norm. In 2009-10, 
however—with the inception of the STEM Academy—attendance rose slightly to 91%, with an 18% 
decrease of suspensions (to 116). Last year (2010-11) saw an even greater increase in attendance (at 
93%) and a further 17% decrease in suspensions (to 
96). The school leadership attributes this increase 
in attendance and decrease in suspensions to the 
engagement of students in the STEM Academy, as 
students take responsibility for learning and have 
pride in themselves and their school. 

Also, the retention rates of students into the 
next year of the STEM Academy are promising. The 
first class, fall 2009, saw an 81% retention into the 
next year. The second cohort saw a 97% retention 
rate into fall 2011. 

The University of Colorado ad¬ 
mitted eight Skyline High School 



First-year CU-Boulder students brainstorm 
on the best solution for improving an 
electrical device for their capstone 
engineering pro ject. 


My role in the Pykata project was to lead and 
organize the programmers, to organize prob¬ 
lems, and help create solutions for programs. 
This project was extremely different than oth¬ 
ers as it was primarily student ied with teacher 
guidance, it was awesome to have Skyline High 
School back us up on our programming de¬ 
velopment. / am honored to have worked with 
the brightest students in the field of computer 
programming at Skyline High School. 

- Meng Koh, Class of 2011 
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graduates to their first-year engineering cohort for the current fall 2011 semester 
(a marked increased from previous years). These newly admitted students report that the presence 
of a serious, focused school culture helped lead them to matriculate to post-high school engineering 
studies. And, CU-Boulder’s presence through the TEAMS Program made students aware of engineer¬ 
ing as a career, and specifically towards engineering at CU-Boulder. Parents and students consider 
this partnership, along with a guaranteed admission to CU-Boulder’s College of Engineering and 
Applied Science for STEM Academy students who meet certain criteria, as a strong argument when 
considering high schools. 


LESSONS LEARNED 


The Skyline STEM Academy is a dynamic process. The combined university and high school lead¬ 
ership team is committed to refining and improving the program through each subsequent year of 
implementation. Lessons learned from year to year help drive improvements in the program and 
increase student learning and success of the Academy. 

The Application Process 

Skyline High School has increased the quality of their application process from their first round 
of fall 2009 applicants. During the inaugural year, the Academy essentially admitted anyone who 
applied and even allowed teachers to verbally recommend some students who did not actually fill 
out the application. Nearly all of those students dropped out of the STEM program—believed to 
be due to a lack of program buy-in (none of those students had actually made the effort to fill out 
the application). The school has since required a more in-depth application process and, has seen 
a higher quality of applicants and student interest. 

The high school still needs to better educate the middle schools on the importance of encour¬ 
aging all of their students to at least investigate the benefits of the Skyline STEM Academy, just as 
they would an IB or AP program at other high schools. While parents are knocking on the school’s 
doors to enroll their students at Skyline, they still have to work hard to get counselors and teachers 
to understand that anyone who is interested can, and should, apply to the STEM Academy. 

Dynamic Teachers 

Recruiting dynamic instructors is always a challenge. How does the school select teachers that 
can maintain the level and methods of instruction that they know works? It helps to have a passion 
about some field in particular. And, teachers in the STEM Academy encourage experimentation and 


SUMMER 2012 


17 






ADVANCES IN ENGINEERING EDUCATION 

The Skyline TEAMS Model: A Longitudinal Look at the Impacts of K-12 
Engineering on Perception. Preparation and Persistence 


students being a part of the daily discussion. The paradigm shift for teachers is that they have become 
facilitators of learning; thus, teachers must be willing and capable of learning alongside the students 
as they all encounter the magical world of engineering. The current Academy teachers fit this mold, 
but more teachers who share these characteristics will be needed as the Academy grows. 

Integration of the Engineering Design Process 

Skyline High School’s four-year STEM curriculum focuses on higher-level thinking, communica¬ 
tion, writing and many other 2T*-century skills necessary for successful student learning. Students 
have expressed that the project team concept that is utilized in all of the STEM classes is essential 
for their learning and development as students prepare to be leaders in today’s global economy. 
From the beginning, students understand the design process and begin to get a feel early on how 
engineers contribute to the health, happiness and welfare of society — an early decision made by 
the leadership team to focus on the engineering design process and embed it into the daily routine 
rather than do “science-fair type” projects in an after-school environment, as many STEM programs 
do. The STEM Academy allows students to design, test, re-design their products and present, and 
communicate their results, and engages students in enriching projects that they work on for many 
weeks or an entire semester—while learning that engineering is meaningful work that benefits hu¬ 
manity and our planet. Through the Academy, students are engaged during the school day, making 
science, technology, engineering and math part of their world every day. 

While the engineering design process remains the consistent focus as students move through 
the program, students are given freedom to choose the type of engineering project that they wish 
to explore. The idea behind the Senior Design Capstone course is that students may pick from top¬ 
ics they have previously explored in their Creative or Advanced Engineering courses. This idea led 
to the concept of the Innovation Center—a center that allows students to work in a real business 
research and development environment, utilizing the Engineering Design process, to find solutions 
to actual, contemporary problems and projects facing industry. 

Female and Minority Retention 

The Skyline High School leadership team has learned that they need to be more purposeful about 
engaging girls and minorities in the STEM Academy program for the entire four years. While STEM 
teachers, guided by CU-Boulder diversity specialists and information gleaned from the National 
Academy of Engineering’s Changing the Conversation study [19], have chosen engineering design 
areas they hope will appeal to girls and minority students more than traditional engineering topics, 
the leadership team must strive more purposefully to engage these groups to maintain their STEM 
interest and commitment through academic achievement. Likewise, the school needs to create a 
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plan to more broadly generate 8 th grade student interest and dispel the notion that only the top 
math and science students should apply to the program—essentially the same challenge that (fast 
forwarded four years) faces engineering colleges interested in broadening participation beyond 
majority men. 

As the word “STEM” was abuzz in the Skyline community last year, the number of student appli¬ 
cants to the Academy greatly increased for fall 2011, but alas the school did not see an increase in 
female and minority students, and is not likely to without a purposeful communications and messag¬ 
ing strategy. With purposeful engineering interactions beginning in their elementary and continuing 
throughout their middle schools, the team intends to positively improve their participation results 
each year. This year, they will invite not only 8 th grade students and parents to their December Open 
House, but also 7 th grade parents and students to garner interest early. They will also work with their 
middle schools more closely to create an integrated STEM curriculum to ensure that their goals and 
strategies around engaging underrepresented populations mirror the high school’s. One of the two 
feeder middle school has recently chosen STEM as their focus, which should greatly aid in the high 
school’s recruitment into the STEM Academy. 

Industry Partners 

The school has realized that cultivating business partnerships is more than asking for donations 
or conducting financial transactions. Toward that end, the Academy leadership regularly meets with 
their industry partners to establish new student/industry collaborations. For example, they have de¬ 
veloped a true industry partnership with a local telecommunications company, RidgeviewTel Wireless, 
that involves their CEO consulting with the school, employing Skyline students, and helping guide 
their program in such a way that their company 
benefits from Skyline’s students and vice versa 
(evidenced through the boxed quote at right). 

Such industry engagement is important to 
the success of a STEM Academy. In March 2011, 

Secretary of the Interior Ken Salazar and other 
federal officials visited the Skyline High cam¬ 
pus to tour the new STEM Academy facility and 
discuss with students their experiences. Salazar 
acknowledges that the partnership between in¬ 
dustry, University, and a K-12 school makes en¬ 
gineering design at the K-12 level possible (see 
video at http://teachers.egfi-k12.org/colorado- 


Working at RidgeviewTel was a wonderful 
experience for me. I learned how to work as a 
part of a team, something that is invaluable in 
the workspace. 

Education had been taught in the same ways 
for generations; however, i feel that Skyline 
has boldly marched into the 21st century by 
teaching with technology in a way the students 
understand and by encouraging collaborative 
efforts to solve a problem. 

- Josh Rahm, Class of 2011 
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stem-salazar/ ) [18], 


CONCLUSION 


Central to the Skyline TEAMS model is their successful partnership with CU-Boulder’s College of 
Engineering and Applied Science, their many industry partnerships and a focus on scholarship and 
achievement for all students. Sophisticated, integrated course work and reinforcing with students 
the choices they must make in order to be academically prepared for a post-secondary engineering 
and/or other STEM education is a main thrust of the STEM Academy. Unfortunately, K-12 engineer¬ 
ing initiatives such as these alone will not create a significant enough STEM pipeline, since a vast 
number of high school students are not academically prepared to enter engineering college. While 
K-12 engineering experiences, such as engineering electives, may inspire an interest in engineering, 
students must also make course selections across the curriculum that adequately prepare them for 
an engineering future. 

Skyline High School’s four-year STEM curriculum is designed to be replicable at other high schools. 
There are several essential elements to reproduce similar systemic and cultural change. The Academy 
is a model that truly emphasizes STEM, requiring four full years of math, science, and engineering 
for all Academy students. Integral to this approach are a focus on 21 st -century skills necessary to 
be leaders in today’s global economy and the academic supports in place to help students choose 
and complete their courses. Students are required to complete Algebra 1 by 9 th grade, and extra 
resource has been designated to help them meet this goal. The STEM Academy also incorporates 
hands-on engineering design courses in every year, cementing the real-world applications of what 
they are learning. This aggressive approach requires more credit hours of courses than the mini¬ 
mum graduation requirements in the district, but the program results indicate that this extra effort 
is worth it. Introducing the program to the elementary and middle school levels is necessary to 
excite and engage students early and is also beneficial for attracting diverse quality students into 
the program. Dedicated leadership, strategic university partnerships, and dynamic teachers all influ¬ 
ence the perceptions, preparation, and persistence in STEM for youngsters that marks the success 
of this STEM Academy. 

Attitude matters! The leadership team at Skyline High School and CU-Boulder believe that all 
students can achieve success at a high level no matter their status, race, ethnicity, or gender. En¬ 
gineering requires upper-level critical thinking, and girls and boys at Skyline are learning that all 
students can do engineering. The STEM Academy provides a venue for students to understand 
what engineers do, explore engineering design, and gain the confidence to pursue post-secondary 
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engineering or other STEM fields. 

After years of school-wide academic and disci¬ 
pline struggles, the Skyline principal finally hears 
passion behind students’ voices as they discuss 
their pursuits in and outside of the classroom 
at a more mature level. They know what they’re 
learning and are proud to talk about it; they have 
become responsible for their education and in¬ 
terested in it. Increasingly, Skyline youth do not 
have barriers on their thinking, and, as such, the 
school is trying its best to harness that concept 
and foster creative student thinking. Students 
participating in the Skyline STEM Academy see their world as “full of opportunities rather than 
barriers.” As evidenced by student presentations to Secretary of the Interior Ken Salazar and rep¬ 
resentatives from Ball Aerospace in the spring of 2011, Skyline STEM students not only want to be 
engineers, they are starting to think like engineers (see boxed quote at right). 

The Academy leadership team values the E in STEM and makes it an integrated part of the day 
and an integral part of the curriculum. And, they have set high expectations for all kids, not just the 
best performers, as all Academy participants are expected to rise to the challenge of becoming a 
“best” performer. The leadership team also shares a consistent vision for achievement which has 
become the norm of the STEM Academy, and spread to the entire school. Everyone who works on 
the team must buy-in to the vision, and everyone works towards the common set of goals. 

And, crucial to the success of the program, the Skyline STEM Academy continues to grow each 
year—from 90 students in the first year to 249 students in the third year (2011-12). With this growth, 
the program needs to continue developing their courses to be engaging, such as the new junior-level 
Advanced Engineering and senior-level Capstone Design course. And, the hope is that CU-Boulder’s 
College of Engineering and Applied Science reaps the benefits of a highly qualified, motivated crop 
of diverse students prepared to pursue an engineering future. 

There are yet many questions that lay ahead in this journey, such as, how to measure a student’s 
ability to think like an engineer? How to inspire the Academy students to continue to achieve aca¬ 
demic excellence throughout their high school experience, and how will engineering colleges change 
to meet the needs of this new student already possessing years of engineering design savvy? The 
Skyline STEM Academy students are starting to recognize the needs in the world around them and 
thinking like engineers. Academy students are beginning to seek out problems and think about how 
to solve new challenges. The Academy instructors hope that the quality of the senior Capstone Design 


At the moment i would love to be a civil 
engineer. But most importantly I would love to 
have a career in which I can make a difference 
by using math, science, and hard work. 

I heard about the STEM program when / was 
in middle school. / was interested in both IB and 
STEM programs. But after trying out IB, I feel 
that STEM would be a better fit for me. 

- 9th Grade STEM Academy Student, 
Class of 2014 
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projects completed through the Innovation Center are relevant enough to industry that a student’s 
value is seen as a viable resource for businesses looking to partner with Skyline High School. The 
Academy works to transition students from wanting to be an engineer, to believing that they are an 
engineer —before they enter a post-secondary institution or the workforce. 
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